Method and Apparatus for Sorting

ABSTRACT

An apparatus for sorting and related methods are described and include acquiring a multiplicity of synchronized image signals of a product stream which is to be sorted; generating a multiplicity of fused sensor signals; forming an image model previously acquired from the objects to be sorted; identifying objects in the product stream, and generating object presence and defect signals; determining a spatial orientation of the objects in the product stream; detecting the defects and removing the defects from the product stream.

RELATED APPLICATIONS

This application is a Continuation application of co-pending U.S. patentapplication Ser. No. 17/945,306, filed Sep. 15, 2022, entitled “A Methodand Apparatus for Sorting,” for which a Notice of Allowance (NOA) hasbeen issued by the USPTO and which is fully and completely incorporatedherein by reference. Co-pending U.S. patent application Ser. No.17/945,306, filed Sep. 15, 2022, entitled “A Method and Apparatus forSorting,” is a Continuation application of U.S. patent application Ser.No. 16/670,179, filed Oct. 31, 2019, entitled “A Method and Apparatusfor Sorting,” which has been issued by the USPTO as U.S. Pat. No.11,446,707, which is a Continuation application of U.S. patentapplication Ser. No. 15/634,694, filed Jun. 27, 2017, entitled “A Methodand Apparatus for Sorting,” which has been issued by the USPTO as U.S.Pat. No. 10,478,863, both of which are fully and completely incorporatedherein by reference. This Continuation patent application also claimspriority to earlier filed PCT/US2018/014362, having the same title,filed on Jan. 19, 2018 and which is fully and completely incorporatedherein by reference.

The inventorship of this Continuation patent application is the same asthe inventorship of co-pending U.S. patent application Ser. No.17/945,306 and of U.S. patent application Ser. Nos. 16/670,179 and15/634,694 and of PCT/US2018/014362 to which this Continuationapplication claims priority.

TECHNICAL FIELD

The present invention relates to a method and apparatus for sorting, andmore specifically to a method for determining a defect in anagricultural product which typically cannot be visually discerned, andthen removing the product having the agricultural defect or the defectitself, from a moving product stream.

BACKGROUND OF THE INVENTION

The developers of optical sorting systems which are uniquely adapted forvisually inspecting a mass-flow of a given food product have endeavored,through the years, to provide increasing levels of information which areuseful in making well-informed sorting decisions to effect sortingoperations in mass-flow food sorting devices. While the capturing andprocessing of product images employing prior art cameras and otheroptical devices has long been known, it has also been recognized thatimages of a product formed by visible spectrum electromagnetic radiationtypically will not provide enough information for an automated sortingmachine to accurately identify all (and especially hidden) food or otheragricultural defects, and which may subsequently be later identifiedafter further processing of the product. For example, one of the defectsin agricultural products which have troubled food processors through theyears has been the effective identification of “sugar end” defects inpotato products, and more specifically potato products that are destinedfor processing into food items such as French fries and the like.

“Sugar ends” and which are also referred to as “dark ends”, “glassyends”, “translucent ends” and “jelly ends” is a physiological, tuberdisorder, which is caused by environmental conditions which occur duringthe growth of the potato plant. Potato strips or fries made from “sugarend” potatoes exhibit or display undesirable dark-brown areas on theproduct after it has been subjected to frying. This defect is typicallycaused by the higher concentration of reducing sugars found in the givendarkened region of the potato. The process of frying the product resultsin caramelizing, which creates the undesirable dark brown region on thefried product. Heretofore, the challenge with food processors has beenthat the “sugar end” defects are typically invisible to traditionaloptical detection technology until after the potato product has beenthoroughly fried. In view of this situation, potato strip processors canbe unaware they have “sugar end” problems with a given lot of potatoesthey are processing until their downstream food service customers frythe potato strips and then provide complaints. “Sugar ends” are usuallyassociated with tubers that have a harvested shape which are somewhatpointed on the stem end of the potato. However, tubers having what isconsidered to be an ideal shape may also develop this anomaly.

Those skilled in the art have recognized that cultural, or managementpractices that increase a potato plant's susceptibility to heat ormoisture stress during tuber initiation, and bulking, can encourage“sugar end” development. As should be understood, tubers areparticularly sensitive to environmental stress during the early bulkingphase. It has been found that sugars can develop in tubers weeks or evenmonths after environmental stress occurs.

Prior art attempts have been made to provide a means for detecting“sugar ends” in an optical sorting device. An example of one of thesedevices is seen in U.S. Patent Publication No. U.S. 2014/0056482A1 toBurgstaller et al. and which discloses a sensor unit in a machine fordetecting “sugar end” defects of potatoes, and which includes amethodology which has the steps of irradiating potatoes with at leastone light source and collecting a reflected light; and then applying atleast one classification feature to the light measurement signals takenfrom the reflected light. In the methodology as described in thatpublished application, the at least one classification featurecorresponds to a predefined “sugar end” criterion. Once the respectivepotato being sorted is classified as having a “sugar end” defect it is,thereafter, removed from further processing. It should be understoodthat the means employed in this published application for thedetermination or detection of a classification feature comprises, atleast in one form of the invention, calculating a deference curve forindividual locus points by calculating the differences between thespectral light measurement signals of the respective locus points, andthe spectral values of a referenced spectrum for a number ofwavelengths; or by calculating the differences between the nthderivative of the spectral light measurement signals of the respectivelocus points, and the nth derivative of the reference spectrum for anumber of wavelengths.

While this methodology, as discussed in the above-referenced publishedpatent application has achieved some degree of success, theimplementation of the methodology has proved, in some instances, to bedifficult or cumbersome. Consequently, the amount of potato productswhich can be processed utilizing this same technology appears to besomewhat limited in view of the complexity of the methodology as morespecifically outlined in that reference. The teachings of U.S. PatentPublication No. U.S. 2014/0056482A1 is incorporated by reference herein.

The present invention, as described hereinafter, avoids the detrimentsassociated with the prior art practices, and provides a new method ofsorting which allows food processors an improved means for detecting,and then removing agricultural products having defects in a manner notpossible, heretofore.

SUMMARY OF THE INVENTION

A first aspect of the present invention relates to a method of sortingwhich includes acquiring a multiplicity of synchronized image signalsfrom a plurality of image generating devices; generating a multiplicityof fused sensor signals by combining the multiplicity of synchronizedimage signals of the image generating devices; forming an image modelcomprising image signals previously acquired from objects of interestand defects; applying the image model to the multiplicity of fusedsensor signals, and forming a resulting object presence and defectsignals; identifying individual objects of interest with the respectiveobject presence and defect image signals; determining a spatialorientation and location of the objects of interest in each of theobject presence and defect image signals; detecting defects within theobject presence and defect image signals by comparing defect aspectsrelative to object aspects, to object images formed of the objectpresence and defect image signal; and removing the unacceptableagricultural products having defects from the product stream.

Another aspect of the present invention relates to a method of sortingwhich includes acquiring a multiplicity of synchronized image signals,each having discreet signal features, from a plurality of imagegenerating devices, and wherein the synchronized image signals representindividual agricultural products traveling in a product stream, andwhich have characteristics, and aspects, which are deemed acceptable forfurther processing, and characteristics, and aspects, which are deemedunacceptable, for further processing; generating a multiplicity of fusedsensor signals by combining the multiplicity of synchronized imagesignals by a selective synchronization of the image generating devices,and by utilizing a known position, orientation, and an operationalresponse of the respective image generating devices so as to allow thegeneration of an accurate spatial resolution of each of the agriculturalproducts travelling in the product stream, and to further align thesignal features of each of the image signals; predicting the presence ofthe acceptable, and unacceptable agricultural products in the fusedsensor signals by applying an image model previously formed from amultiplicity of image signals acquired from acceptable and unacceptableagricultural products, to the multiplicity of fused sensor signals, soas to facilitate the formation of a resulting acceptable agriculturalproduct image signal; and an unacceptable agricultural product imagesignal; identifying individual agricultural products travelling in theproduct stream as being an acceptable, or an unacceptable agriculturalproduct, by identifying one or more of a group of pixels in each of theacceptable and unacceptable agricultural product image signals;determining a spatial orientation of the identified individualagricultural products travelling in the product stream by applying aprior source of knowledge of acceptable and unacceptable agriculturalproduct aspects, characteristics and agricultural object images, to theacceptable and unacceptable agricultural product image signals;detecting unacceptable agricultural products by applying a prior sourceof knowledge of unacceptable agricultural product aspects and objectimages to the acceptable and unacceptable agricultural product imagesignals; identifying the location of the unacceptable agriculturalproducts in the acceptable and unacceptable product image signals; andremoving the unacceptable agricultural products from the product stream.

These and other aspects of the present invention will be discussed ingreater detail hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the invention are described below withreference to the following accompanying drawings.

FIG. 1 is a greatly simplified, schematic view of an apparatus which maybe utilized to practice the present methodology.

FIG. 2 is a greatly simplified, schematic view of the methodology of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

This disclosure of the invention is submitted in furtherance of theconstitutional purposes of the U.S. Patent Laws “to promote the progressof science and useful arts” (Article 1, Section 8).

The present methodology of the invention is generally indicated by thenumeral 10, and is best understood by a study of FIGS. 1 and 2 ,respectively. As best understood by a study of FIG. 1 , the methodemploys, in one form of the invention, a conveying device which isfragmentarily shown in FIG. 1 as a conveying device 11. The conveyingdevice, as illustrated in that drawing, shows a distal, discharge end 12of a conveyor belt 11, and which further has an upper conveying surface13 which supports a product stream 14 to be inspected. The productstream 14 is formed of individual objects of interest 15, which arereleased from the distal, discharge end 12, and allowed to move, underthe influence of gravity, in a downwardly directed path of travel whichis generally indicated by the numeral 16. A source of the objects ofinterest or products which is to be inspected, and sorted, is labeled bythe numeral 17.

The product stream 14 has objects of interest or products 15 which, inone form of the invention, may include various agricultural productswhich have both acceptable features for further processing 20, orunacceptable features for processing which are generally indicated bythe numeral 21. For example, in the processing of potatoes, unacceptablefeatures 21 of a potato product would be the presence of “sugar ends” orregions of rot, which will be detected by the methodology as described,hereinafter. Positioned downstream of the distal discharge end 12, ofthe conveying device 11, are a multiplicity of image capturing devices22 which are generally shown, and which further are positioned laterallyoutwardly relative to the downwardly directed path of travel 16. Themultiplicity of imaging capturing devices 22 (which may include, forexample hyperspectral or multispectral cameras of assorted designs) areutilized, in a first step of the present method, and which includesacquiring a multiplicity of synchronized image signals 23 each havingdiscreet signal features, from a plurality of image generating devices22. The multiplicity of image capturing devices 22 produce amultiplicity of synchronized image signals 23 which are then selectivelysupplied to a first controller, and which is further generally indicatedby the numeral 24. The synchronized image signals 23 representindividual objects of interest 15 such as agricultural products whichare traveling in the product stream 14, and which have characteristicsand aspects which are deemed acceptable 20, for further processing, andcharacteristics and aspects which are deemed a defect, or unacceptable21 for further processing. The downwardly directed path of travel 16 ofthe product stream 14 passes through a downstream inspection stationwhich is generally indicated by the numeral 25, and a downstream defectremoval station 26, and which is further located elevationally, belowthe inspection station 25. The multiplicity of image capturing devices22 are positioned so as to acquire image signals 23 from the objects ofinterest 15 while they pass through the inspection station 25 in amanner well understood in the art. Assorted optical reflectors 27, andoptical combiners 28, are provided, and which co-align multiple imagecapturing devices 22. Further, well known background elements areprovided, and which additionally are positioned laterally, outwardlyrelative to the product stream 14, and which is passing through theinspection station 25.

As best seen by reference to FIG. 1 , the methodology of the presentinvention includes a step of generating a source of synchronizedelectromagnetic radiation 31 which is directed towards, and reflected atleast in part from, the product stream 14, and which further is formedof the individual objects of interest 15 passing through the inspectionstation 25. As seen in FIG. 1 , an electromagnetic radiation emitter 30is generally illustrated, and which is positioned laterally, outwardlyrelative to the inspection station 25. When selectively energized, theelectromagnetic radiation emitter emits a source of electromagneticradiation 31 which is directed towards, and reflected at least in partby the individual objects of interest 15 passing through the inspectionstation 25. This emitter of electromagnetic radiation may producevarious wavelengths of electromagnetic radiation which enhances theability of the respective image capturing devices 22 to captureindividually unique images of the objects of interest 15 which allow forthe identification of assorted characteristics and aspects of eachobject of interest, and in particular, various defects in the productstream 14. For example, the defects could include “sugar ends” orregions of rot which are present in a potato products undergoing theinspection and sorting. As also seen in FIG. 1 , an electromagneticradiation emitter control signal 32 is provided to each electromagneticradiation emitter 30. This control signal 32 causes the selectiveenergizing of each of the emitters 30. The selective energizing of theindividual electromagnetic radiation emitters 30 to achieve theaforementioned benefits of this invention, is explained in significantdetail in U.S. Pat. No. 9,266,148, the teachings of which areincorporated by reference herein. The method 10 of the present inventionincludes still another step of generating a multiplicity of fused imageand sensor signals 34 by combining the multiplicity of synchronizedimage and sensor signals 23 by a selective synchronization of the imageand sensor generating devices 22 and the respective electromagneticradiation emitters 30, and by utilizing a known position, orientationand an operational response of the respective image and sensorgenerating devices 22, so as to allow the generation of an accuratespatial resolution of each of the objects of interest 15 which aretraveling in the product stream 14, and to further align the signalfeatures of each of the image and sensor signals. As seen in FIG. 1 ,the control signal 32 is operably controlled by the selectivesynchronization step 34. The selective synchronization 34 of the imageand sensor signals and the energizing of the electromagnetic radiationemitters 30, is performed in a manner so as to inhibit the destructiveinterference that may occur when the electromagnetic radiation emitters30, and sensors 22 are simultaneously rendered operable. In themethodology 10 of the present invention, the synchronized image andsensor signals 23 are formed by a methodology which includes the step ofselecting image signals 23 from multiple, different sensors 22. Stillfurther in the methodology 10 of the present invention the aligning ofthe signal features of each of the synchronized image and sensor signalsso as to form, at least in part, the multiplicity of fused sensorsignals 35 comprises the step of conducting a spatial registration ofthe sensor signals 23 with each other, and with an ejector controller100, and which will be discussed in greater detail, below. The highaspect spatially fused sensor signals 35 are then supplied to the firstcontroller 24 as seen in FIG. 2 .

The high aspect spatially fused sensor and image signals 35 are providedto the controller 24, and to individual modules within the controller24, (FIG. 2 ) and which perform many of the methodology steps of thepresent invention. In this regard, the first controller 24 has a module40 which contains prior knowledge of object defects which have beenacquired from the previous inspection of similar objects of interest.This prior knowledge of the object defects 40 is provided to anothermodule 41, and which conducts supervised training of the firstcontroller 24, so as to allow the present methodology to teach itselfhow to improve the sorting reliability of the present invention 10. Thesupervised training module 41 is supplied with a portion of the highaspect, spatially fused sensor and image signals 35, and which has beenacquired from the multiple image capturing devices 22. Still further thefirst controller 24 includes a prediction module 42 which predicts thepresence of the objects of interest 15, and possible defects in thefused sensor signals 35 by applying an image model 43, which waspreviously formed from a multiplicity of image and sensor signals whichwere acquired from the objects of interest 15, and the defects, to themultiplicity of fused sensor signals 35, so as to facilitate theformation of a resulting object presence image signal 44, and defectimage signal 45, respectively. The present methodology 10 includes asecond controller 50.

The second controller 50 is operably coupled with the first controller24. Still further the second controller has a module 60 which implementsa step in the methodology 10 which includes identifying the individualobjects of interest 15 with the object presence signals 44, anddefective image signals 45, by identifying one, or more of a group ofpixels in each of the object presence and defect image signals. Stillfurther, the second controller 50, and more specifically the module foridentifying objects of interest 60, is operable to supply a signal 61 tothe module for supervised training 41, so as to allow the module forsupervised training 41 to continue to learn as the inspection processproceeds so as to increase the accuracy and sorting efficiency of thepresently disclosed methodology 10. The object presence and defectsignals 44 and 45 are supplied to other modules in the second controller50. More specifically, the second controller 50 has a module forimplementing a step which includes storing and supplying a source ofknowledge of the object aspects for use in the sorting process. Thismodule 70 supplies the stored information to another module 71, andwhich implements a step in the methodology 10 of determining a spatialorientation of the objects of interest 15 traveling in the productstream 14 by applying the prior source of knowledge of the objectaspects 70, to a multiplicity of object images which are formed of theobject presence and defect image signals 44 and 45, respectively. Stillfurther the second controller 50 includes a module which provides aprior source of knowledge of defect aspects 80, relative to objectaspects. In this regard this prior knowledge 80 is provided to a module81 for detecting defects within the unacceptable objects of interest 15by applying the prior source of knowledge 80 to object aspects 70, andto the object images formed of the object presence and defect signals 44and 45, respectively. The module for detecting defects 81, and theobject presence, and defect signals 44 and 45, generates a defect signalwhich is generally indicated by the numeral 82, and which further isitself supplied to an object removal control 90. The object removalcontrol 90 generates a signal 91 which is provided to an ejectorcontroller, which further is generally indicated by the numeral 100.Therefore, the methodology of the present invention 10 after identifyingthe location of the unacceptable objects of interest having defects 81,in the object image signals, the methodology 10 includes a step ofremoving the unacceptable objects of interest having defects from theproduct stream by means of the ejector controller 100. The ejectorcontroller operably controls an air manifold 101, (FIG. 1 ) ofconventional design, and which is further configured to release apressurized blast of ambient air 102, which is operable to removedefective objects of interest 21 from the downwardly descending productstream 16, so as to form a product stream having only acceptable objectsof interest 15.

Operation

The operation of the described embodiment of the present invention isbelieved to be readily apparent, and is briefly summarized at thispoint.

In its broadest aspect the present invention relates to a method ofsorting 10 which comprises a first step of acquiring a multiplicity ofsynchronized image signals 23 of individual objects of interests 15, anddefects 21, from a plurality of image generating devices 22. The methodincludes another step of generating a multiplicity of fused sensorsignals 34 by combining the multiplicity of synchronized image signals23 of the image generating devices 22. Still further the method ofsorting 10 of the present invention includes yet another step of formingan image model 43 comprising image signals 23 which were previouslyacquired from the objects of interest 15, and the defects 20. The methodincludes yet another step of applying the image model 43, to themultiplicity of fused sensor signals 34, and forming a resulting objectpresence 44, and defect signals 45, respectively. The method of thepresent invention 10 includes another step of identifying individualobjects of interest 60, with the respective object presence 44, anddefect image signals 45. The method of the present invention includesyet another step of determining a spatial orientation and location ofthe objects of interest 71 in each of the image signals 23. The methodincludes yet another step of detecting defects 81 within the objectpresence and defect signals 44 and 45, respectively, by comparing defectaspects 80 relative to object aspects 70, to object images formed of theobject presence and defect image signals 44 and 45, respectively.Finally, the present invention in its broadest aspect includes a laststep of removing 100 the objects of interest 15 having defects 21 fromthe product stream 14.

The method 10 of the present invention includes another step, andwherein the synchronized image signals 23 represent individual objectsof interest 15 such as agricultural products traveling in a productstream 14, and which have characteristics, and aspects, which are deemedacceptable for further processing 20, and characteristics and aspectswhich are deemed unacceptable for further processing 21. This is bestseen in FIG. 1 . The method includes another step, and wherein thesynchronized image signals 23 are formed by a selective synchronization23 of the image generating devices 22, and by utilizing a knownposition, orientation and an operational response of the respectiveimage generating devices 22 so as to allow the generation of anaccurate, spatial resolution of each of the objects of interest 15traveling in the product stream 14, and to further align the signalfeatures of each of the image signals 23. In the present invention themethod includes a further step, and wherein the image models 43 areformed from a methodology which includes previously acquiring 40 amultiplicity of image signals 23 from known acceptable 20, andunacceptable 21 objects of interest 15, such as agricultural products,and the like. In the present methodology 10 the step of identifyingindividual objects of interest 60 further includes the step ofidentifying one or more pixel groups in each of the object presence 44,and defect image signals 45 which identify the acceptable objects ofinterest 20 or defects 21.

In the present invention the methodology 10 includes another step ofdetermining the spatial orientation 71, and location of the respectiveobjects of interest 15 and further comprises another step of developinga prior source of knowledge of object aspects 70, and which is appliedto object images which are formed of the object presence and defectsignals 44 and 45, respectively. In the present methodology 10, the stepof detecting unacceptable agricultural products, or defects in theobjects of interest 81, further comprises another step of developing aprior source of knowledge of defect aspects 80, relative to objectaspects 70, to a multiplicity of object images formed of the objectpresence and defect image signals 44 and 45, respectively. In thepresent methodology 10, and before the step of acquiring themultiplicity of synchronized image signals 23, the method furthercomprises still another step of providing a product stream 14 ofindividual objects of interest 15, such as agricultural products havingboth acceptable agricultural products 20, and unacceptable products 21,and which must be removed from the product stream 14. The methodincludes another step of passing the product stream 14 having both theacceptable agricultural products 20 and unacceptable agriculturalproducts or objects of interest 21 through an inspection station 25(FIG. 1 ). In the present invention, the methodology 10 includes anotherstep of acquiring the multiplicity of synchronized image signals 23, andafter the step of providing the product stream 14, generating a sourceof synchronized electromagnetic radiation 31 which is directed towardsand reflected, at least in part, from the product stream 14 of theagricultural products or objects of interest passing through theinspection station 25. In the present invention the method 10 furthercomprises a step of providing a first controller 24 which predicts thepresence of the objects of interest 15 and defects 21 in the fusedsensor signals 35, and which further applies the image model 43 to atleast some of the multiplicity of fused sensor signals 35. The methodfurther comprises another step of providing a second controller 50, andwhich identifies individual objects of interest 15, and defects 21, inthe object presence and defect image signals 44 and 45; and furtherdetermines the spatial orientation 71 of the identified objects ofinterest 15 and defects 21 traveling in the product stream 14. Themethod 10 also includes another step of identifying the defects 81, andthe defect image signal 82 identifies the location of the defects in thedefect image signals. An unacceptable agricultural product image signalis provided to the third controller or object removal control 90. In thepresent methodology 10, the third controller 90 is controlled, andcoupled with, and renders operational an ejector 101. The thirdcontroller 90 is coupled in signal receiving relation relative to theunacceptable agricultural product image or defect signal 82 which isgenerated by the second controller 50, and further renders the ejector101 operational by means of the ejector controller 100 to remove theunacceptable agricultural product or any defects 21 from the productstream 14, and which is passing by, or through, the defect removalstation 26.

More specifically the methodology 10 of the present invention furtherincludes a step of acquiring a multiplicity of synchronized image andsensor signals 23, each having discreet signal features, from aplurality of image generating and sensor devices 22. The synchronizedimage and sensor signals 23 represent individual objects of interest 15,such as agricultural products, which are traveling in a product stream14, and which have characteristics and aspects which are deemedacceptable for further processing 20, and characteristics and aspectswhich are deemed a defect and unacceptable 21 for further processing.The present method includes another step of generating a multiplicity offused image and sensor signals 34 by combining the multiplicity ofsynchronized image and sensor signals 23 by a selective synchronizationof the image and sensor generating devices 22, and by utilizing a knownposition, orientation, and an operational response of the respectiveimage and sensor generating devices 22 so as to allow the generation ofan accurate spatial resolution 35 of each of the objects of interest 15traveling in the product stream 14, and to further align the signalfeatures of each of the image and sensor signals 23. The method includesstill another step of predicting the presence 42 of the objects ofinterest 15, and possible defects 21, in the fused image and sensorsignals 35 by applying an image model 43 which is previously formed froma multiplicity of image signals 35, and which are further acquired fromthe objects of interest 15 and defects 21, to the multiplicity of fusedsensor signals 35 so as to facilitate the formation of a resultingobject presence image signal 44, and a defect image signal 45. Themethod 10 includes another step of identifying the individual objects ofinterest 15, with the object presence and defect image signals 44 and45, by identifying one or more of a group of pixels in each of theobject presence and defect image signals 44 and 45, respectively. Themethod 10 of the present invention further includes another step ofdetermining a spatial orientation 71 of the objects of interest 15traveling in the product stream 14 by applying a prior source ofknowledge 70 of the object aspects to a multiplicity of the objectimages which are formed of the object presence and defect image signals44 and 45, respectively. The method includes yet another step ofdetecting defects 81 within the unacceptable objects of interest 15 byapplying a prior source of knowledge 80 of defect aspects relative toobject aspects, to the object images formed of the object presence andobject defect signals 44 and 45, respectively. The method includes stillanother step of identifying the location 81 of the unacceptable objectsof interest having defects in the object image signals; and yet anotherstep 100 of removing the unacceptable objects of interest 15 havingdefects from the product stream 14 so as to provide a resulting uniformproduct stream.

In the methodology of the present invention 10 the discreet signalfeatures of the multiplicity of synchronized image signals 23 areselected from the group comprising signals generated by any one or moreof individual hyperspectral or multispectral imagers or scanners 22which are employed in the apparatus and which are schematicallyrepresented in FIG. 1 .

Still further the synchronized image signals 23 are formed by amethodology which includes a step of conducting a spatial registrationof the respective image signals. In addition to the foregoing, theaspects and characteristics of the objects of interest 15, and which aredeemed acceptable for further processing are selected from individualproducts 15 having a known, and acceptable qualities. Still further, thecharacteristics of the objects of interest 15 which are deemedunacceptable for further processing are selected from the groupcomprising individual products 15 having known unacceptable qualities.Moreover, the aligning of the signal features of each of thesynchronized image signals 23 so as to form, at least in part, themultiplicity of fused image or sensor signals 35 comprises another stepof conducting a spatial registration of the respective sensors 22 witheach other, and with the ejector controller 100.

The methodology 10 of the present invention further includes yet othersteps which are directed to the formation of the image model 43. In thisregard, the image model 43 is formed by a methodology which includes astep of utilizing a standard classification algorithm such as a partialleast square algorithm (PLS). In addition to the foregoing, the priorsource of knowledge of the object aspects 70, which is supplied to themultiplicity of object images, and which are further used to determinethe spatial orientation of the identified objects of interest 71 isformed by a methodology which includes the steps of conducting an objectshape analysis; and conducting an object aspect measurement. Moreover,the prior source of knowledge of the defect aspects 80, and which areapplied to the multiplicity of object images formed by the presentmethodology comprises the step of qualifying unacceptable pixel groupsfound in the image signals with object regions identified in objectaspects. Additionally, the step of removing the unacceptable objects ofinterest 90 from the product stream 14 further comprises the step ofremoving an unacceptable portion of an object of interest 15, from anacceptable portion of the same object of interest 15.

As should be understood, and in the present methodology 10, and beforethe step of acquiring the multiplicity of synchronized image signals 23,the method 10 further includes a step of providing a product stream 14of individual objects of interest which have characteristics and aspectsof both acceptable 20, and unacceptable objects of interest 21, andpassing the product stream 14 having both the acceptable andunacceptable objects of interest 20 and 21 through an inspection station25. In addition to the foregoing the methodology further includes,before the step of acquiring the multiplicity of synchronized imagesignals 23, and after the step of providing the product stream 14,generating a source of synchronized electromagnetic radiation 31 whichis directed towards, and reflected at least in part from, the productstream 14 which is formed of the objects of interest 15 passing throughthe inspection station 25. In addition to the foregoing, the method ofthe present invention 10 includes yet another step of providing a firstcontroller 24 which predicts the presence of the objects of interest 15,and defects 21 in the fused sensor signals 35, and which further appliesthe image model 43 to the multiplicity of fused sensor and image signals35. The method 10 further includes still other steps of providing asecond controller 50 which identifies individual objects of interest 15,and defects 21, in the product stream; determines the spatialorientation of the identified individual objects of interest 71traveling in the product stream 14; detects the objects of interest 15;identifies the location of the defects 21 in the object presence anddefect image signals 44 and 45, and further generates a signal 82 whichindicates the presence and location of the defect(s) 21 in the productstream 14. The method 10 of the present invention includes yet anotherstep of providing a defect removal station 26, and positioning thedefect removal station downstream of the inspection station 25 (FIG. 1). The method includes another step of providing an ejector 101, andpositioning the ejector 101 in the defect removal station 26, and whichis effective, when made operational, to remove the unacceptable objectsof interest 21 from the product stream 14. The method 10 includes yetanother step of providing an ejector controller 100 which iscontrollably coupled with, and renders operational the ejector 101. Athird controller 90 is coupled in defect signal 82 receiving relationrelative to the second controller 50, and which further renders theejector controller 100 operational to cause the ejector 101 to removethe unacceptable objects of interest 21 from the product stream 14 (FIG.1 ). Therefore it will be seen that the present invention provides aconvenient means for detecting unacceptable objects of interest in aproduct stream, and which may include agricultural products such aspotatoes having agricultural defects such as “sugar ends,” in aparticularly efficient fashion which has not been available in automatedsorting devices utilized, heretofore. The present methodology isefficient, operates reliably, and further has an inventive feature whichallows it to self-learn so as to increase the efficiency and reliabilityof the sorting operations of a device employing same.

In compliance with the statute, the invention has been described inlanguage more or less specific as to structural, and methodicalfeatures. It is to be understood, however, that the invention is notlimited to the specific features shown and described since the meansherein disclosed comprise preferred forms of putting the invention intoeffect. The invention is, therefore, claimed in any of its forms ormodifications within the proper scope of the appended claimsappropriately interpreted in accordance with the Doctrine ofEquivalence.

We claim:
 1. A method of sorting comprising: providing a product streamof individual objects of interest and passing the product stream throughan inspection station; acquiring a multiplicity of synchronized imagesignals of the individual objects of interest from hyperspectral ormultispectral image generating devices, and wherein the acquiredmultiplicity of synchronized image signals represent the individualobjects of interest traveling in the product stream, and thecharacteristics and aspects of the individual objects of interest, whichare deemed acceptable for further processing, and the characteristicsand aspects of the individual objects of interest, which includefeatures that are undetectable to traditional optical detection, andwhich are deemed unacceptable, for further processing; providing animage model formed from a multiplicity of previously acquiredsynchronized image signals from individual objects of interest havingacceptable characteristics and aspects, and from individual objects ofinterest having unacceptable characteristics and aspects that includefeatures that are undetectable to traditional optical detection;generating a multiplicity of fused image and sensor signals by combiningthe multiplicity of synchronized image signals by selectivesynchronization of the hyperspectral or multispectral image generatingdevices, and by utilizing an operational response of each of thehyperspectral or multispectral image generating devices to generate aspatial resolution of each of the individual objects of interesttravelling in the product stream; providing a controller that implementsa control function; providing a first control function to predict thepresence of acceptable, and unacceptable characteristics in theindividual objects of interest in the product stream by applying theimage model to the multiplicity of fused image and sensor signals, tofacilitate formation of an object presence image signal, and tofacilitate formation of a defect image signal; providing a secondcontrol function to identify individual objects of interest travellingin the product stream as being an acceptable agricultural product, or asbeing an unacceptable agricultural product, by identifying one or moreof a group of pixels in each of the object presence image signals anddefect image signals which identify objects of interest or defects;detecting defects within the unacceptable individual objects of interestin the product stream by applying a prior source of knowledge ofunacceptable object of interest aspects relative to acceptable object ofinterest aspects to object images formed of the object presence imagesignals and the defect image signals; updating the image model withadditional and subsequently acquired object presence image signals anddefect image signals; and providing a third control function whichrenders operational an ejector, and wherein the third control functionis coupled in signal receiving relation relative to an unacceptabledefect signal which is generated by the second control function, andwhich further renders the ejector operational to remove unacceptableobjects of interest from the product stream.
 2. A method as claimed inclaim 1, wherein each of the multiplicity of synchronized image signalshas discreet signal features.
 3. A method as claimed in claim 1, whereina known position and orientation of each of the hyperspectral ormultispectral image generating devices is used to generate the spatialresolution of each of the individual objects of interest traveling inthe product stream.
 4. A method as claimed in claim 1, wherein theoperational response of each of the hyperspectral or multispectral imagegenerating devices is further used to align the discrete signal featuresof each of the multiplicity of synchronized image signals.
 5. A methodas claimed in claim 3, wherein the known position and orientation andthe operational response of each of the hyperspectral or multispectralimage generating devices is further used to align the discrete signalfeatures of each of the multiplicity of synchronized image signals.
 6. Amethod as claimed in claim 3, further comprising: determining a spatialorientation and location of the identified individual objects ofinterest travelling in the product stream by applying a prior source ofknowledge of acceptable agricultural product characteristics and aspectsand unacceptable agricultural product characteristics and aspects, tothe object presence image signals and to the defect image signals.
 7. Amethod as claimed in claim 6, further comprising: identifying theposition and location of the unacceptable individual objects of interestin the product stream.
 8. A method of sorting, comprising: acquiring amultiplicity of synchronized image and sensor signals from a pluralityof image generating devices and sensors, and wherein the synchronizedimage and sensor signals represent individual objects of interest whichare traveling in a product stream, and which have characteristics andaspects which are deemed acceptable for further processing, andcharacteristics and aspects which are deemed a defect and which deemeddefects include features that are undetectable to traditional opticaldetection, and unacceptable, for further processing; generating amultiplicity of fused sensor signals by combining the multiplicity ofsynchronized image and sensor signals by a selective synchronization ofthe image generating devices and sensors, and by utilizing anoperational response of the respective image generating devices andsensors so as to allow the generation of a spatial resolution of each ofthe objects of interest travelling in the product stream; predicting thepresence of the objects of interest, and possible defects in the fusedsensor signals by applying an image model previously formed from amultiplicity of image signals which were acquired from the objects ofinterest, and the defects, to the multiplicity of fused sensor signalsso as to facilitate the formation of a resulting object presence imagesignal and a defect image signal; identifying the individual objects ofinterest with the object presence, and defect image signals, byidentifying one or more of a group of pixels in each of the objectpresence, and defect image signals; detecting defects withinunacceptable objects of interest by applying a prior source of knowledgeof defect aspects relative to object aspects, to the object imagesformed of the object presence, and defect signals; and removing theunacceptable objects of interest having defects from the product stream.9. A method as claimed in claim 8, wherein each image and sensor signalhas discreet signal features.
 10. A method as claimed in claim 8,wherein a known position and orientation of each of the hyperspectral ormultispectral image generating devices is used to generate the spatialresolution of each of the individual objects of interest traveling inthe product stream.
 11. A method as claimed in claim 8, wherein theoperational response of each of the hyperspectral or multispectral imagegenerating devices is further used to align the discrete signal featuresof each of the multiplicity of synchronized image signals.
 12. A methodas claimed in claim 10, wherein the known position and orientation andthe operational response of each of the hyperspectral or multispectralimage generating devices is further used to align the discrete signalfeatures of each of the multiplicity of synchronized image signals. 13.A method as claimed in claim 10, further comprising: determining aspatial orientation and location of the identified individual objects ofinterest travelling in the product stream by applying a prior source ofknowledge of acceptable agricultural product characteristics and aspectsand unacceptable agricultural product characteristics and aspects, tothe object presence image signals and to the defect image signals.
 14. Amethod as claimed in claim 13, further comprising: identifying theposition and location of the unacceptable individual objects of interestin the product stream.
 15. An apparatus for sorting comprising: aconveying device having a conveying surface, the conveying device beingconfigured such that a product stream of individual objects of interestcan be conveyed along the conveying surface, one or more of theindividual objects of interest have features that are undetectable totraditional optical detection, said features that are undetectable totraditional optical detection are deemed acceptable or unacceptable forfurther processing; an inspection station having a plurality ofhyperspectral or multispectral image generating devices, the inspectionstation positioned relative to the conveying device such that theproduct stream passes through the inspection station, each of theplurality of hyperspectral or multispectral image generating devices isconfigured to generate a multiplicity of synchronized image signals ofthe individual objects of interest, and wherein the multiplicity ofsynchronized image signals represent the individual objects of interesttraveling in the product stream; wherein the multiplicity ofsynchronized image signals are combined by selective synchronization ofthe hyperspectral or multispectral image generating devices to generatea multiplicity of fused image and sensor signals, and a spatialresolution of each of the individual objects of interest travelling inthe product stream is generated by utilizing an operational response ofeach of the hyperspectral or multispectral image generating devices; acontroller that implements a plurality of control functions, saidplurality of control functions includes a first control function topredict the presence of acceptable and unacceptable characteristics inthe individual objects of interest in the product stream by applying animage model to the multiplicity of fused image and sensor signals, tofacilitate formation of an object presence image signal, and tofacilitate formation of a defect image signal, the image model is formedfrom a multiplicity of previously acquired synchronized image signalsfrom individual objects of interest having acceptable characteristicsand aspects, and from individual objects of interest having unacceptablecharacteristics and aspects that include features that are undetectableto traditional optical detection, said plurality of control functionsincludes a second control function to identify individual objects ofinterest travelling in the product stream as being an acceptable objectof interest or as being an unacceptable object of interest, byidentifying one or more of a group of pixels in each of the objectpresence image signals and defect image signals which identify objectsof interest or defects; wherein defects within the unacceptableindividual objects of interest in the product stream are detected byapplying a prior source of knowledge of unacceptable individual objectsof interest characteristics and aspects relative to acceptableindividual objects of interest characteristics and aspects to objectimages formed of the object presence image signals and the defect imagesignals; and said plurality of control functions includes a thirdcontrol function which renders operational an ejector, and wherein thethird control function is coupled in signal receiving relation relativeto an unacceptable individual objects of interest defect image signalwhich is generated by the second control function, and which furtherrenders the ejector operational to remove unacceptable individualobjects of interest from the product stream.
 16. The apparatus asclaimed in claim 15, wherein each of the multiplicity of synchronizedimage signals has discreet signal features.
 17. The apparatus as claimedin claim 15, wherein a known position and orientation of each of thehyperspectral or multispectral image generating devices is used togenerate the spatial resolution of each of the individual objects ofinterest traveling in the product stream.
 18. The apparatus as claimedin claim 15, wherein the operational response of each of thehyperspectral or multispectral image generating devices is further usedto align the discrete signal features of each of the multiplicity ofsynchronized image signals.
 19. The apparatus as claimed in claim 17,wherein the known position and orientation and the operational responseof each of the hyperspectral or multispectral image generating devicesis further used to align the discrete signal features of each of themultiplicity of synchronized image signals.
 20. The apparatus as claimedin claim 17, wherein a spatial orientation and location of theidentified individual objects of interest travelling in the productstream is determined by applying a prior source of knowledge ofacceptable individual objects of interest characteristics and aspectsand unacceptable individual objects of interest characteristics andaspects, to the object presence image signals and to the defect imagesignals.
 21. The apparatus as claimed in claim 20, wherein the positionand location of the unacceptable individual objects of interest in theproduct stream are identified.
 22. The apparatus as claimed in claim 15,wherein the image model is updated with additional and subsequentlyacquired object presence image signals and defect image signals.
 23. Anapparatus for sorting comprising: a plurality of individual objects ofinterest traveling in a product stream, wherein the individual objectsof interest have characteristics and aspects which are deemed acceptablefor further processing, and one or more of the individual objects ofinterest have characteristics and aspects which are deemed a defect andwhich deemed defects include features that are undetectable totraditional optical detection, and unacceptable for further processing;and an inspection station comprising a plurality of image generatingdevices and sensors, the plurality of image generating devices andsensors are configured to generate a multiplicity of synchronized imageand sensor signals and wherein the synchronized image and sensor signalsrepresent the individual objects of interest traveling in the productstream; wherein a multiplicity of fused sensor signals are generated bycombining the multiplicity of synchronized image and sensor signals by aselective synchronization of the image generating devices and sensors,and by utilizing an operational response of the respective imagegenerating devices and sensors so as to allow the generation of aspatial resolution of each of the individual objects of interesttravelling in the product stream; wherein the presence of the individualobjects of interest and possible defects is indicated in the fusedsensor signals by applying an image model previously formed from amultiplicity of image signals which were acquired from the objects ofinterest, and the defects, to the multiplicity of fused sensor signalsso as to facilitate the formation of a resulting object presence imagesignal, and a defect image signal; wherein the individual objects ofinterest are identified with the object presence and defect imagesignals, by identifying one or more of a group of pixels in each of theobject presence and defect image signals; wherein defects withinunacceptable objects of interest are detected by applying a prior sourceof knowledge of defect aspects relative to object aspects, to the objectimages formed of the object presence and defect signals; and wherein theunacceptable objects of interest having defects are removed from theproduct stream.
 24. The apparatus as claimed in claim 22, wherein eachof the multiplicity of synchronized image signals has discreet signalfeatures.
 25. The apparatus as claimed in claim 22, wherein a knownposition and orientation of each of the hyperspectral or multispectralimage generating devices is used to generate the spatial resolution ofeach of the individual objects of interest traveling in the productstream.
 26. The apparatus as claimed in claim 22, wherein theoperational response of each of the hyperspectral or multispectral imagegenerating devices is further used to align the discrete signal featuresof each of the multiplicity of synchronized image signals.
 27. Theapparatus as claimed in claim 25, wherein the known position andorientation and the operational response of each of the hyperspectral ormultispectral image generating devices is further used to align thediscrete signal features of each of the multiplicity of synchronizedimage signals.
 28. The apparatus as claimed in claim 25, wherein aspatial orientation and location of the identified individual objects ofinterest travelling in the product stream is determined by applying aprior source of knowledge of acceptable individual objects of interestcharacteristics and aspects and unacceptable individual objects ofinterest characteristics and aspects, to the object presence imagesignals and to the defect image signals.
 29. The apparatus as claimed inclaim 28, wherein the position and location of the unacceptableindividual objects of interest in the product stream are identified. 30.The apparatus as claimed in claim 22, wherein the image model is updatedwith additional and subsequently acquired object presence image signalsand defect image signals.